Search for: All records

While Deep Reinforcement Learning has emerged as a de facto approach to many complex experience-driven networking problems, it remains challenging to deploy DRL into real systems. Due to the random exploration or half-trained deep neural networks during the online training process, the DRL agent may make unexpected decisions, which may lead to system performance degradation or even system crash. In this paper, we propose PnP-DRL, an offline-trained, plug and play DRL solution, to leverage the batch reinforcement learning approach to learn the best control policy from pre-collected transition samples without interacting with the system. After being trained without interaction with systems, our Plug and Play DRL agent will start working seamlessly, without additional exploration or possible disruption of the running systems. We implement and evaluate our PnP-DRL solution on a prevalent experience-driven networking problem, Dynamic Adaptive Streaming over HTTP (DASH). Extensive experimental results manifest that 1) The existing batch reinforcement learning method has its limits; 2) Our approach PnP-DRL significantly outperforms classical adaptive bitrate algorithms in average user Quality of Experience (QoE); 3) PnP-DRL, unlike the state-of-the-art online DRL methods, can be off and running without learning gaps, while achieving comparable performances. 

Aqueous electrolytes typically suffer from poor electrochemical stability; however, eutectic aqueous solutions—25 wt.% LiCl and 62 wt.% H₃PO₄—cooled to −78 °C exhibit a significantly widened stability window. Integrated experimental and simulation results reveal that, upon cooling, Li⁺ions become less hydrated and pair up with Cl⁻, ice‐like water clusters form, and H⋅⋅⋅Cl⁻bonding strengthens. Surprisingly, this low‐temperature solvation structure does not strengthen water molecules’ O−H bond, bucking the conventional wisdom that increasing water's stability requires stiffening the O−H covalent bond. We propose a more general mechanism for water's low temperature inertness in the electrolyte: less favorable solvation of OH⁻and H⁺, the byproducts of hydrogen and oxygen evolution reactions. To showcase this stability, we demonstrate an aqueous Li‐ion battery using LiMn₂O₄cathode and CuSe anode with a high energy density of 109 Wh/kg. These results highlight the potential of aqueous batteries for polar and extraterrestrial missions.

 

Aqueous electrolytes typically suffer from poor electrochemical stability; however, eutectic aqueous solutions—25 wt.% LiCl and 62 wt.% H₃PO₄—cooled to −78 °C exhibit a significantly widened stability window. Integrated experimental and simulation results reveal that, upon cooling, Li⁺ions become less hydrated and pair up with Cl⁻, ice‐like water clusters form, and H⋅⋅⋅Cl⁻bonding strengthens. Surprisingly, this low‐temperature solvation structure does not strengthen water molecules’ O−H bond, bucking the conventional wisdom that increasing water's stability requires stiffening the O−H covalent bond. We propose a more general mechanism for water's low temperature inertness in the electrolyte: less favorable solvation of OH⁻and H⁺, the byproducts of hydrogen and oxygen evolution reactions. To showcase this stability, we demonstrate an aqueous Li‐ion battery using LiMn₂O₄cathode and CuSe anode with a high energy density of 109 Wh/kg. These results highlight the potential of aqueous batteries for polar and extraterrestrial missions.